Skip to content

Conversation

@ioanatia
Copy link
Contributor

We see a bunch of new failures for multi clusters

#127268
#127279
#127304
#127272

All these failures happened in the last 24h - most likely there are somehow related to #126705 which was merged last week.
However I am not sure why we have not seen these types of failures earlier.

Most likely all FORK related tests will fail with the same type of error for multi clusters - I haven't seen any for multi or single node tests.
I am skipping the FORK tests for multi clusters for now.

@ioanatia ioanatia added >non-issue >test Issues or PRs that are addressing/adding tests Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch :Search Relevance/Search Catch all for Search Relevance labels Apr 24, 2025
@ioanatia ioanatia requested a review from carlosdelest April 24, 2025 09:10
Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, but should we instead use muted-tests.yml and open an issue so we keep track of this?

@ioanatia
Copy link
Contributor Author

I'm OK with this, but should we instead use muted-tests.yml and open an issue so we keep track of this?

I'm using the same pattern we have for INLINESTATS and unmapped fieds:

assumeTrue("Test " + testName + " is skipped on " + oldVersion, isEnabled(testName, instructions, oldVersion));
assumeFalse("INLINESTATS not yet supported in CCS", testCase.requiredCapabilities.contains(INLINESTATS.capabilityName()));
assumeFalse("INLINESTATS not yet supported in CCS", testCase.requiredCapabilities.contains(INLINESTATS_V2.capabilityName()));
assumeFalse("INLINESTATS not yet supported in CCS", testCase.requiredCapabilities.contains(JOIN_PLANNING_V1.capabilityName()));
assumeFalse("INLINESTATS not yet supported in CCS", testCase.requiredCapabilities.contains(INLINESTATS_V5.capabilityName()));
assumeFalse("LOOKUP JOIN not yet supported in CCS", testCase.requiredCapabilities.contains(JOIN_LOOKUP_V12.capabilityName()));
// Unmapped fields require a coorect capability response from every cluster, which isn't currently implemented.
assumeFalse("UNMAPPED FIELDS not yet supported in CCS", testCase.requiredCapabilities.contains(UNMAPPED_FIELDS.capabilityName()));

FORK might be used when testing RRF and RERANK (if I am not mistaken, there's a test for using RERANK with FORK and RRF). Until we figure out why CCS suddenly has flaky specs, I don't want to make it a requirement when adding a test that's using FORK that's not in the fork.csv-spec to mute it in muted-tests.yml.

Figuring out the CCS failures is tracked in #121950

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/Search Catch all for Search Relevance Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test Issues or PRs that are addressing/adding tests v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants